REMARKS 



This AMENDMENT UNDER 37 CFR 1.1 1 1 is filed in reply to the outstanding Office 
Action of April 19, 2004, and is believed to be fully responsive thereto for reasons set forth 
below in greater detail. 

Pursuant to paragraphs 1 -4 of the Office Action, a marked up version and a clean 
version of a substitute specification are submitted. 

The undersigned attorney states that the substitute specification contains no new 
matter, as many of the insertions are present in the original priority document and were 
omitted apparently for some computer translation reason, and the substitute specification also 
corrects apparent errors such as changing a "1" to an — n— . 

Responsive to paragraphs 5-8 of the Office Action, the claims have been reviewed and 
amended to conform with 35 USC 112. 

Responsive to paragraph 7, claims 7 and 21, this limitation is explained at page 4, 
lines 22-24, wherein "configuration" means a "relative arrangement of parts or elements as 
(1) SHAPE," as defined in Webster's Ninth New Collegiate Dictionary, p. 275 enclosed. 

Responsive to paragraph 7, claims 8-9 and 22-23, the "basic patterns corresponding to 
said constituent data of said image blocks" are explained in the specification at page 10, lines 
1-6, as follows. 

"The image processing apparatus employs the key data Key to correspond individual 
bits wh, of the watermark data W representing such additional information as copyright 
information, with two basic patterns Pha and Phb of m (m> 2) types of basic patterns PI to Pm 
that have the same structure as the image blocks Iij but have different contents. The two basic 
patterns Pha and Phb (Pha, Phb) correspond to values (1, 0) of the bits wh of the watermark 
data W..." 
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Responsive to paragraph 8 of the Office Action, "claims are generally narrative," all of 
the claims have been reviewed and amended where appropriate to be less narrative and to 
better explain the subject matter of the claims while conforming with 35 USC 112. 

Reconsideration is respectfully requested of the rejections in paragraphs 10-16 of the 
claims herein over the prior art, including a primary reference Oami et al U.S. 6,415,041, 
either alone or in combination with the secondary references of Zhao et al U.S. 6,141,753, and 
Ogawa et al U.S. 6,704,431, and Kita et al U.S. 6,707,927, and Hobson et al U.S. 6,633,653. 

The present invention has the following basic distinctions and advantages relative to 
the primary reference Oami et al., as supplied by the inventor. 

The present invention provides a watermarking method that calculates a correlation 
coefficient between a watermarked image and a predefined watermark pattern that provides 
concealed information. As may be seen in Figure 4, the predefined watermark pattern is 
composed of 8 by 8 pixel sized (size is JPEG block size) patterns, which is referred to as basic 
patterns. 

Each pixel of a basic pattern has +1 or -1 or 0 value. Since we (invention) use a 
correlation coefficient for the detection, the predefined pattern can be modified in a positive 
direction, pixel by pixel, to embed the predefined watermark pattern. Therefore, we add a 
positive value to a pixel in the original image such that the corresponding watermark pixel 
value is +1. 

The present invention modifies a predefined watermark pattern in an invisible and 
robust manner, and also is very fast and can be implemented with affordable equipment. 

Claim 1 specifies the method of generating the embedding pattern from the predefined 
pattern. Let's say P is a basic block patten for a processing block in the original image, PI, 
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P_2, P_k is generated from P in advance in the manner as described above (enhance the 
pixel value of P in the positive direction). This process is depicted by Fig. 7. 

P_l, ... P_k, may be created quasi-randomly or intuitively from P. In this example, the 
term "plurality of candidate data" refers to P_l, P_2, ... ,P_k, and "corresponding data 
embedded into object data" refers to P. Then one of P_l, P_k is selected based on an 
invisibility factor (variation indication data in claim 1) generated by a "variation indication 
data generation means", and a detectability factor (detectability indication data in claim 1) 
generated by a "detectability indication data generation means". 

One distinction of the present invention over the reference Oami et al is that the 
invention does not categorize an input image. Instead, the invention calculates the affect of 
embedding a selected pattern through the "variation indication data generation means". This is 
a more general and practical solution than categorizing an input image into a predefined 
range. 

Another related distinction is that Oami's method categorizes an input image based 
upon the entire image. In contrast, the present invention processes an input image block wise 
(see FIG.6), and probably can embed more information (in bits), and realize a more precise 
quality control. 

This distinction over Oami is now expressly claimed in the added first subparagraph in 
each of independent claims 1, 2, 13, 14, 15 and 16. The added first subparagraph in each of 
the independent claims is fully supported in the specification at page 9, lines 15-18, of the 
original specification. 

There is a similarity between Oami and the present invention in the sense that both 
methods select an embedding pattern from a plurality of patterns, but as described above, the 
patterns of the present invention are block based and each block pattern is originated from a 
basic pattern . 
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The inventor has also supplied the following comments on differences between 
Oami's watermarking method and the method of the present invention: 

1 . Oami's method has a frequency domain watermark pattern (described in US641 5041 
column 1), which means watermark pattern in real domain, or pixel domain, spreads over 
entire image, whereas our watermark pattern is composed from block patterns with 8 by 8 
pixel size or different block size. 

2. Oami's method controls the strength of the watermark pattern (described in US641 504, 
column 2, line 24-26). It does not control pattern pixel-wise. In this sense, Oami's method 
may have a difficulty in balancing image quality and robustness against attack. 

3. Oami's method requires the original image for detecting information (their method is 
based on the method described in IEEE TRANS. ON IMAGE PROCESSING, Vol 6, 
No.12, 1997, pp 1673-1687, which is referred at US641504, column 1, line 13). On the 
contrary, our method just requires a key to obtain embedded information. 

FIG 5 in the Oami patent describes creating their "digital watermark characteristic 
table", which is the pre-process required before embedding watermark into images. 
Here it creates a map from the image category, watermark strength to image degradation 
degree, and resistant value against attacking. 

FIG 5 is distinct from the present invention as follows: 

1 . Oami's method is interested in selecting strength of the watermark , which affects the 
entire image quality. On the contrary, our method is interested in selecting a pattern from 
candidate patterns. Candidate patterns are originated from a basic pattern and selection finds 
the best fit pattern for the image of the currently processed block, and only affects the block. 

2. Oami's method uses images to create a "digital watermark characteristic table". The 
image used is not the image in the watermarking process depicted by FIG. 1 . Instead Oami's 
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method uses a category to represent the image. On the contrary, our method uses a 
watermarking image to obtain quality degradation. 

Oami et al explain in column 1 1 , lines 40-44, that the image-quality degradation 
degree calculation section compares the input image (corresponding to the object data) to the 
watermarked image output (corresponding to the object data obtained by embedding each of 
the plurality of candidate data) to determine the degradation amount between the two images 
(corresponding to the variation indication data). Oami et al further explain in column 7, lines 
47-58, that multiple watermarks of different strength (corresponding to S(l), S(2) ... S(M) are 
each embedded as a plurality of candidate data into the input image data. 

S(l), S(2), . . . S(M) describe a relationship between a given parameter "strength 
constraint", 1,. . M 5 and an observed watermark strength for the image category k. 
On the contrary, our candidate data are patterns generated from a basic pattern as described 
above, and they are not related to strength constraint. Please note that Oami et al does not 
explain the particular method they use(i for image quality control. Our invention describes the 
method to adjust a predefined pattern to fit in the given image, and does not have a strength 
constraint which is provided for adjustment of image quality. 

The secondary references do not compensate for and obviate the basic distinctions and 
advantages of the present invention relative to Oami et al, and accordingly a detailed 
discussion of the secondary references is not believed to be warranted herein. 

This application is now believed to be in condition for allowance, and a Notice of 
Allowance is respectfully requested. If the Examiner believes a telephone conference might 
expedite prosecution of this case, it is respectfully requested that he call applicant's attorney 
at (516) 742-4343. 



Scully, Scott, Murphy & Presser 
400 Garden City Plaza 
Garden City, New York 1 1530 
(516) 742-4343 
WCR:jf 




Respectfully submitted. 



William C. Roch 
Registration No. 24,972 
Attorney for Applicants 
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DATA PROCESSING APPARATUS. IMAGE PROCESSING 
APPARATUS. AND METHOD THEREFOR 

BACKGROUND OF THE INVENTION 
Technical Field 

[0001] The present invention relates to a data processing apparatus, an image 

processing apparatus and a method therefor, for embedding additional information, such 
• as copyright information, in object data, such as image data or audio data. More 

particularly, the present invention is directed to a data processing apparatus, an image 
processing apparatus and a method therefor, for appropriately selecting from a plurality 
of candidate data sets, in accordance with object data contents, optimal watermark data 
that while ordinarily indiscernible can be easily detected, and for embedding the 
watermark data in the object data. 

! 
i 

Prior Art 

[0002] In International Publication No. WO 97/49235, for example, a system is 

disclosed for employing pixel block coding (PBC) to embed copyright information (also 

i 

generally referred to, hereinafter, as verification data) in object data, such as image data, 
so that it is not visually discernible (the system for embedding verification data in content 
data so that it is not visually discernible is also, hereinafter, called an "electronic 
watermarking system). 





\ 



[0003] In International Publication No. WO 98/1 1 6928 a method is disclosed for 

employing the electronic watermarking system disclosed in document 1 to inhibit the 
alteration of image data and to effectively protect literary materials. 



[0004] In Japanese Unexamined Patent Publication No. Hei 10-164549, an 

improvement, to the electronic watermarking method in document 1, is disclosed for 
verifying indivisible embedded watermark data in order to detect the alteration of image 
data. In addition to these documents, an invention concerning the electronic 
watermarking system is disclosed in Japanese Unexamined Patent Publications Nos. Hei 
09-151747, Hei 10-83310, Hei 10-106149, Hei 10-161933, Hei 10-164349, Hei 10- 
285562, Hei 10-334272, Hei 10-240626 and Hei 10-240129, and USP 5,315,448 
(Japanese Patent No. Hei 8-507912), USP 5,745,604, USP 5,768,426, USP 5,850,481 and 
USP 5,832,119. 

[0005] However, excluding one part of these documents, a method is not 

disclosed for optimizing watermark data in accordance with the contents of object data, 
e.g., the quality of the images provided by image data, and for embedding the optimized 
watermark data. 

[0006] Further, while one part of these documents carries a description 

concerning a method, in accordance with the contents of an image, for changing 

watermark data that is to be embedded, the methods disclosed in the documents are less 

compatible with the properties of the data that are to be embedded, and thus, the 

watermark data can not be satisfactorily optimized. 
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[0007] To resolve the conventional problem, it is highly desirable to provide a 

data processing apparatus, and a method therefor, for optimizing watermark data that 
indicate the presence of additional information, such as copyright information, in 
accordance with the contents of object data, and for embedding the optimized watermark 
data. 

[0008] It is further highly desirable to provide an image processing apparatus, and 

a method therefor, for optimizing watermark data in accordance with the quality of image 
data, such as the property of an object, so that watermark data that is less noticeable but is 
easily detectable can be employed to perform embedding. 

SUMMARY OF THE INVENTION 

[0009] To achieve the above objects, the present invention is directed to a data 

processing apparatus selecting one of a plurality of candidate data corresponding to 
watermark data embedded into object data to embed the selected candidate data as the 
watermark data, comprises: a variation indication data generation means for generating a 
plurality of variation indication data indicating variation between the object data and each 
of the object data obtained by embedding each of the plurality of the candidate data; a 
detectability indication data generation means for generating a plurality of detectability 
indication data each indicating how easily each of the plurality of candidate data being 
detected; a watermark data selection means for selecting one of the candidate data based 
on the plurality of variation indication data and the plurality of detectability indication 
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data; and a data embedding means for embedding the selected candidate data into the 
object data as the watermark data. 

[0010] The present invention is further directed to an image processing apparatus 

selecting one of a plurality of candidate data corresponding to watermark data embedded 
into object image data to embed the selected candidate data into the object image data as 
the watermark data, comprising: a variation indication data generation means for 
generating a plurality of variation indication data indicating variation between the image 
data and each of the image data obtained by embedding each of the plurality of the 
candidate data; a detectability indication data generation means for generating 
detectability indication data each indicating how easily each of zero or more candidate 
data being detected; a candidate data selection means, for employing the detectability 
indication data to select one of the candidate data that corresponds to variation indication 
data for variations that are smaller than a predetermined reference; and a data embedding 
means for embedding the selected candidate data as the watermark data in the image data. 

[0011] Preferably, the detectability indication data generation means generates 

detectability indication data indicating the detectability of the candidate data 
corresponding to the variation indication data indicating variations smaller than the 
predetermined reference. 

[0012] Also, when no candidate data corresponding to the variation indication 

data for variations smaller than the predetermined reference exists, preferably, the 
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candidate data selection means selects predetermined supplement data, instead of the 
candidate data. 

[0013] In addition, each of the object image data is each of a plurality of image 

blocks obtained by dividing one image data set, and each of the watermark data 
corresponding to each of the object image data is each of one or more types of constituent 
data constituting additional information that is added to the object image data. The image 
processing apparatus further comprises: a watermark data correspondence means for 
corresponding the constituent data constituting the additional information data with the 
plurality of image blocks, as the watermark data; and a candidate data generation means 
for generating the plurality of candidate data corresponding to the constituent data 
corresponded with the plurality of image block. 

[0014] Preferably, the watermark data correspondence means accepts 

predetermined key data to correspond the constituent data of the additional information 
data with the plurality of image blocks based on the predetermined key data. 

[0015] Preferably, the candidate data generation means generates, as the plurality 

of candidate data, a plurality of additional patterns employing the same configuration as 
the object image data to be added to the object image data. 

[0016] Also, the candidate data generation means preferably generates the 

plurality of additional patterns by multiplying a plurality of predetermined coefficients 
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with basic patterns corresponding to the constituent data corresponded with the image 
blocks. 

[0017] In addition, the detectability indication data generation means preferably 

calculates the detectability indication data representing a correlation between the 
additional patterns and the basic patterns. The candidate data selection means selects, 
from among the additional patterns, a pattern corresponding to detectability indication 
data representing the highest correlation. The image processing apparatus further 
comprises: a watermark data detection means for detecting the watermark data embedded 
into the image block, based on the correlation of the basic patterns and an image block 
into which the selected additional pattern embedded. 

[00181 Preferably, the watermark data correspondence means sorts the plurality of 

image blocks into one or more of groups, each of which including one or more of the 
image blocks to correspond the constituent data with the image blocks that are included 
in the groups. 

[0019] Also, the variation indication data generation means preferably calculates 

each of differences between each of entropy values for the object image data and each of 
entropy values for the object image data obtained by embedding each of the plurality of 
candidate data as the variation indication data. 
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[0020] In addition, the detectability indication data generation means preferably 

generates the detectability indication data for the respective candidate data corresponding 
to the variation indication data with their values within a predetermined range. 

[0021] The present invention is still further directed to a data processing method, 

for selecting one of a plurality of candidate data corresponding to watermark data 
embedded into object data for embedding the selected candidate data as the watermark 
data, comprising: a variation indication data generation step of generating a plurality of 
variation indication data indicating variation between the object data and each of the 
object data obtained by embedding each of the plurality of the candidate data; a 
detectability indication data generation step of generating a plurality of detectability 
indication data each indicating how easily each of the plurality of candidate data being 
detected; a watermark data selection step of selecting one of the candidate data based on 
the plurality of variation indication data and the plurality of detectability indication data; 
and a data embedding step of embedding the selected candidate data into the object data 
as the watermark data. 

[0022] Even further, the present invention is directed to an image processing 

method, for selecting one of a plurality of candidate data corresponding to watermark 

data embedded into object image data for embedding the selected candidate data into the 

object image data as the watermark data, comprising: a variation indication data 

generation step of generating a plurality of variation indication data indicating variation 

between the image data and each of the image data obtained by embedding each of the 

plurality of the candidate data; a detectability indication data generation step of 
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generating detectability indication data each indicating how easily each of zero or more 
candidate data being detected; a candidate data selection step of employing the 
detectability indication data to select one of the candidate data that corresponds to 
variation indication data for variations that are smaller than a predetermined reference; 
and a data embedding step of embedding the selected candidate data as the watermark 
data in the image data. 

[0023] Still even further, the present invention provides a first storage medium on 

which a computer-readable program selecting one of a plurality of candidate data 
corresponding to watermark data embedded into object data to embed the selected 
candidate data as the watermark data, the computer-readable program permitting a 
computer to perform: a variation indication data generation step of generating a plurality 
of variation indication data indicating variation between the object data and each of the 
object data obtained by embedding each of the plurality of the candidate data; a 
detectability indication data generation step of generating a plurality of detectability 
indication data each indicating how easily each of the plurality of candidate data being 
detected; a watermark data selection step of selecting one of the candidate data based on 
the plurality of variation indication data and the plurality of detectability indication data; 
and a data embedding step of embedding the selected candidate data into the object data 
as the watermark data. 

[0024] The present invention further provides, a second storage medium on which 

a computer-readable program selecting one of a plurality of candidate data corresponding 

to watermark data embedded into object image data to embed the selected candidate data 
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into the object image data as the watermark data, the computer-readable program 
permitting a computer to perform: a variation indication data generation step of 
generating a plurality of variation indication data indicating variation between the image 
data and each of the image data obtained by embedding each of the plurality of the 
candidate data; a detectability indication data generation step of generating detectability 
indication data each indicating how easily each of zero or more candidate data being 
detected; a candidate data selection step of employing the detectability indication data to 
select one of the candidate data that corresponds to variation indication data for variations 
that are smaller than a predetermined reference; and a data embedding step of embedding 
the selected candidate data as the watermark data in the image data. 

BRIEF DESCRIPTIONOF THE DRAWINGS 

[0025] Preferred embodiments of the present invention will now be described, by 

way of example only, with reference to the accompanying drawings in which: 

[0026] Fig. 1 is a diagram showing the arrangement of an image processing 

system that carries out an image processing method according to the present invention. 

[0027] Fig. 2 is a diagram showing the hardware arrangement (a computer) for a 

watermarking unit, a detector and a server in Fig. 1 . 

[0028] Fig. 3 is a diagram showing the structure of a watermark program that is 

executed by the watermarking unit in Fig. 1 in order to perform a watermark data 

embedding process in accordance with the present invention. 
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[0029] Figs. 4A and 4B are specific diagrams showing the image division process 

performed by an image divider in Fig. 3, with Fig. 4A showing the image blocks Iij 
obtained by the image division, and Fig. 4B showing the structure of each image block 

[0030] Fig. 5 is a diagram showing the structure of a basic pattern processor in 

Fig. 3. 

[0031] Fig. 6 is a diagram showing the grouping of the image blocks Iij by a 

grouping unit in Fig. 5. 

[0032] Fig. 7 is a diagram showing a method used by an adjusted basic pattern 

calculator in Fig. 3 to generate adjusted basic patterns P'hijl to P'hijn. 

[0033] Fig. 8 is a diagram showing the arrangement of a watermark pattern 

generator in Fig. 3. 

[0034] Fig. 9 is a diagram showing a method whereby a selector in Fig. 8 selects, 

from adjusted basic patterns P'hijl to P'hij±n, the "1" adjusted basic patterns P"'hijln to 
P m hij! that evidence little variation, even when added to the image block Iij. 

[0035] Fig. 10 is a diagram showing the structure of a detection program that is 

executed by the detector in Fig. 1 to perform a watermark data detection process 

according to the present invention. 
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[0036] Fig. 1 1 is a flowchart (S20) showing the processing performed for the 

watermark program in Fig. 3. 

[0037] Fig. 12 is a flowchart (S40) showing the processing performed for the 

watermark data detection program in Fig. 10. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS OF THE INVENTION 

[0038] An image processing apparatus according to the present invention (also 

referred to, hereinafter, as this image processing apparatus) is the data processing 
apparatus for this invention, and is especially employed for the process for embedding 
watermark data in image data. To aid in the understanding of the invention, the image 
processing apparatus will be described by using specific examples wherein, to the extent 
possible, the same reference numerals are used as are employed in the preferred 
embodiment. The technical scope of the invention, however, is not limited to the specific 
examples. 

[0039] This image processing apparatus divides input image data Vin, consisting 

of H pixels and W pixels, into a plurality of image blocks Iij, each composed of wb x hb 
(e.g., 8x8) pixels, and employs these blocks as object data for the performance of the 
embedding process (the object data are referred to, hereinafter, as image blocks Iij). 
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[0040] Furthermore, based on key data Key, which has an arbitrary data length, 

the image processing apparatus sorts the obtained image blocks Iij into groups SI to Sg, 
which correspond respectively to g bits wl to wg 5 constituting watermark data W having 
an arbitrary data length that represents additional information, such as copyright 
information. 

[0041] Preferably, the image blocks Iij that are included in these groups Sh (1 

[[<]] h [[<]] g) are evenly distributed throughout the image data Vin, and do not overlap 
the other image blocks Iij that are included in the other group Sx (h g x), while the 
same number of image blocks Iij as are included in the other group Sx, which belongs to 
the groups Sh. 

[0042] The image processing apparatus employs the key data Key to correspond 

individual bits wh, of the watermark data W representing such additional information as 
copyright information, with two basic patterns Pha and Phb of m (m [[>]] 2) types of 
basic patterns PI to Pm that have the same structure as the image blocks Iij but have 
different contents. 

[0043] The two basic patterns Pha and Phb (Pha g [[£]] Phb) correspond to the 

values (1 , 0) of the bits wh of the watermark data W, and in accordance with the value of 
the bit wh, one of the basic patterns Pha and Phb is corresponded with the individual bits 
wh of the watermark data W (see the preferred embodiment for the detailed 
correspondence). 
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[0044] Further, the basic patterns Pha and Phb that are corresponded with the bits 

wh may or may not overlap patterns Pxa and Pxb, which are corresponded with the other 
bits wx; however, generally, it is preferable that the basic patterns Pha and Phb be 
selected so that they do not overlap the others. 

[0045] The image block Iij, which is corresponded with one of the bits wh, may 

or may not overlap an image block Iij 1 , which is corresponded with another bit wh\ 

[0046] This image processing apparatus corresponds the bits wh, of the 

watermark data W, with which the basic patterns Ph are corresponded, with the groups Sh 
obtained above. 

[0047] As a result, based on the key data Key, the same pattern Phij (the selected 

basic pattern) and the bits wh of the watermark data W are corresponded for all the image 
blocks Iij included in the group Sh, and corresponded with each image block Iij, are one 
bit Wh and one type of basic pattern (the selected basic pattern in this embodiment) Phij. 

[0048] Next, the image processing apparatus of this invention performs a 

predetermined process, e.g., an orthogonal transformation (DFT is hereinafter used as an 
example), for the image blocks Iij, and generates DFT transformation coefficients I*ij 
(also referred to, hereinafter, simply as image blocks Fij) for the image blocks Iij. 



[0049] Furthermore, the image processing apparatus adds the square of the real 

number and the square of the imaginary number of each element in each image block Tij, 
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and obtains a square root F'ij (referred to, hereinafter, as a power element I"ij or an image 
block Tij). 

[0050] In the following explanation, the DFT is employed as the transformation; 

however, as a modification, another transformation method, such as a DCT (discrete 
cosine transformation) or a wavelet transformation, may be employed. 

[0051] The image processing apparatus multiplies each of the DFT coefficients 

P'hij (also referred to, hereinafter, as basic patterns P'ij) of the basic patterns Phij by, for 
example, the identical elements of n types of coefficients (adjustment coefficients) CI to 
Cn, which have the same structure as the image blocks Iij and each element of which has 
a value of 0 to 1 . Thus, n types of candidate data (adjustment basic patterns in this 
embodiment) P'hij 1 to P'hijn are obtained. The image processing apparatus performs the 
same process as is performed for the image blocks I'ij for the n types of candidate data 
PTiijl to P'hijn, and obtains the power elements P"hijl to P ?, hijn (also referred to, 
hereinafter, as candidate data sets P"hijl to P"hijn). 

[0052] The image processing apparatus of this invention calculates the entropy 

for [e(r , ij+P"hijl) to e(I"ij+P"hijn)] for values (I"ij+P"hijl) to (I"ij+P"hijn) obtained by 
adding the image blocks I"ij and candidate data P ?, hijl to P"hijn. Then, the image 
processing apparatus calculates the absolute values of the differences between the 
calculated entropy values and the entropy e(I'ij) for the image blocks Tij, and regards 
them as variation indication data sets (comparison result data sets) Rl to Rn. 
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[0053] The image processing apparatus employs as a reference a predetermined 

threshold value e, extracted from the candidate data sets P'hijl to P'hijn, and selects 
candidate data sets P m hijl to P m hijl n, which correspond to variation indication data sets 
Rl to Rl that hold a value equal to or smaller than the threshold value e (it should be 
noted that 0 [[<]] 1 [[<]] n; and when 1 = 0, there is no variation indication data equal to or 
smaller than the threshold value e). 

[0054] The variation indication data are not necessarily calculated using the 

power elements of the DFT coefficients, and may be calculated using other elements, 
such as real number elements of the DFT coefficients. Furthermore, entropy need not 
always be used to calculate the variation values (of indiscernibility) when the candidate 
data are embedded in an image block, and another appropriate calculation method may be 
employed. 

[0055] As is described above, the correspondence of the image blocks Iij and the 

basic patterns Phij is uniquely determined by the key data Key, the sum (referred to, 
hereinafter, as an inner product) is acquired for the products obtained by multiplying the 
individual elements of each image block Iij, in which the watermark data are embedded, 
and the individual elements of each basic pattern Phij, and the correlation of these inner 
products is examined to detect the watermark data. Therefore, to embed watermark data 
by adding one of the candidate data sets P m hijl to P m hijn to the DFT coefficients I'ij of 
the image blocks Iij, one of the candidate data sets P" ! hijl to P m hijln that is most nearly 
correlated with the basic pattern P'hij should be employed to easily detect the watermark 
data. 
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[0056] The image processing apparatus calculates the inner products (detection 

capability data in this embodiment) Dl to Din for the basic pattern P'hij and the candidate 
data sets P m hijl to P m hijln, and selects the candidate data set corresponding to the largest 
inner product as the final candidate data set (an embedding pattern in this embodiment) 
P f ij for the image block Iij. 

[0057] Further, the image processing apparatus embeds the watermark data by 

adding identical elements in the image block Tij and in the selected final candidate data 
P f ij, and generates the resultant image data Vw. 

[0058] In this example, the correlation between the basic pattern and the 

candidate data is employed as the scale for detectability. When another detection method 
is employed, only the method used to determine the detectability of the candidate data 
must be changed in accordance with the pertinent detection method. 

[0059] When there are no candidate data that provide variation indication data 

that are equal to or smaller, for example, than the predetermined threshold value e, a 
random pattern (supplemental pattern) Ps, which is prepared in advance, is selected as the 
final candidate data P'ij and is added to the image block I'ij, and the resultant data are 
embedded. 
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[0060] The processing for handling the image data has been employed as an 

example; however, it is apparent that the image processing apparatus of this invention can 
be appropriately modified for the processing for other data. 

[0061] An explanation will now be given, while referring to the above described 

example, for the relationship of the individual components of the image processing 
apparatus of the invention and the components shown in the preferred embodiment, and 
the operation of the individual components. 

[0062] The watermark data correspondence means corresponds to an image 

divider 200 and a basic pattern processor 22 in the preferred embodiment. 

[0063] The watermark data correspondence means accepts the key data Key that 

are used for embedding and detecting watermark data W, and the watermark data W that 
represent additional information, such as copyright information. Based on the key data 
Key and the watermark data W, the watermark data correspondence means uniquely 
corresponds, with each of the image blocks Iij, one of the bits wh that constitute the 
watermark data W and the basic pattern (selected basic pattern) Phaij or Phbij, which 
corresponds to the bit wh and which has a bit value of 0 or 1 . 

[0064] The candidate data generation means corresponds to a basic pattern 

adjustment unit 26 in the preferred embodiment. The candidate data generation means 
multiplies the elements of the basic pattern P'hij, which the watermark data 
correspondence means corresponded with the image block Iij, by the identical elements 
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of a plurality of coefficients (adjustment coefficients) CI to Cn, and generates a plurality 
of candidate sets (adjustment basic patterns) P'hijl to P'hijn. 

[0065] In the image processing apparatus of the invention, the variation indication 

data generation means corresponds to entropy calculators 280 and 284, and an adder 282 
and a comparator 286 in accordance with the preferred embodiment. For the variation 
indication data, the entropy of the image block I"ij and the entropy of the values obtained 
by adding the image block Fij and each of the candidate data sets P"hijl to P"hijn are 
calculated. The absolute values of the differences provided by entropy are obtained for 
each of the variation indication data sets (comparison data sets) Rl to Rn that represent 
the variations due to the embedding of the candidate data. 

[0066] As the entropy variations become smaller, accordingly the variations in 

the image block due to the embedding of the candidate data are also reduced. And when 
the values of the variation indication data are small, the candidate data are indiscernible 
when they are embedded in an image block. 

[0067] The detectability indication data generation means corresponds to a 

detectability calculator 292 in the preferred embodiment. The detectability indication 
data generation means calculates the accumulated addition values (referred to, 
hereinafter, as inner products) obtained by multiplying the elements of the basic pattern 
P'hij, which is corresponded with the image block Iij, and the identical elements of the 
candidate data sets P f "hij 1 to P"'hijln, which provide the variation indication data having 
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values equal to or smaller than the predetermined threshold value e. These inner products 
are regarded as detectability indication data sets Dl to Din. 

[0068] As is described above, the watermark data that is embedded in the 

individual image blocks is detected by obtaining the correlation of the image block Iij and 
each of the basic patterns Pha and Phb, which correspond to the watermark data wh 
assigned for the individual image blocks. 

[0069] Therefore, it is found that of the candidate data sets P'"hij 1 to P ,M hijln, the 

candidate data that provides the basic pattern P'hij having the maximum inner product 
can be detected most easily. 

[0070] The watermark data selection means corresponds to selectors 290 and 296 

in the preferred embodiment. The watermark data selection means compares the 
variation indication data sets Rl to Rn with the threshold value e 5 and selects the 
candidate data that correspond to indication data that are equal to or smaller than the 
threshold value. 

[0071] The detectability indication data generation means calculates the 

detectability indication data sets Dl to Din for the individual candidate data sets P m hijl 
to P m hijln selected by the watermark data selection means. 
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[0072] Further, as the final candidate data (watermark pattern) P'ij for the image 

block Iij, the watermark data selection means selects one of the candidate data sets P'"hijl 
to P'"hijln that corresponds to the maximum detectability indication data. 

[0073] When in this manner the watermark data selection means selects the 

candidate data at two stages, the calculations required of the detectability indication data 
generation means can be reduced. 

[0074] When the calculations need not be taken into account, both the variation 

indication data generation means and the detectability indication data generation means 
can calculate the variation indication data and the detectability indication data for all the 
candidate data sets P'hijl to P ? hijn, and the watermark data selection means can 
simultaneously employ these two data sets to select the candidate data P'ij. 

[0075] The data embedding means corresponds to a pattern embedding unit 204 

in the preferred embodiment. 

[0076] The data embedding unit embeds the selected final candidate data 

(watermark pattern) P f ij in the image block Iij by adding the coefficient of the final 
candidate data P'ij to the identical coefficient of the image block I'ij. 

[0077] The data detection means corresponds to a watermark data detection 

program 40 in the preferred embodiment. 
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[0078] Based on the key data Key, for example, the data detection means 

corresponds the image blocks Tij+P'ij with the basic patterns Phaij and Phbij, which 
respectively correspond to the bit values 0 and 1 . 

[0079] Further, the data detection means obtains the correlations between the 

image block Iij and the basic patterns Phaij and Phbij, and employs the correlations to 
detect the individual bits of the watermark data W. 

[0080] Fig. 1 is a diagram showing the configuration of an image processing 

system 1 that carries out an image processing method according to the present invention. 

[0081] Fig. 2 is a diagram showing the hardware arrangement used for a 

watermarking unit 2, a detector 4 and a server 12 in Fig. 1. 

[0082] As is shown in Fig. 1, the image processing system 1 is so designed that 

the watermarking unit 2, the server 12 and the detector 4 are connected via a data 
transmission network, such as an ISDN network, a telephone communication network, or 
via a data communication network 10 such as a LAN. 

[0083] As is shown in Fig. 2, the watermarking unit 2, the server 12 and the 

detector 4 in Fig. 1 are constituted by a computer 16. 

[0084] The computer 16 comprises: a main body 160, including a CPU 162, a 

memory 164 and their peripheral circuits; a display device 166, such as a CRT display; an 

input device 168, such as a keyboard or a mouse; a communication unit 170 for 
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exchanging data with another node across the network 10; and a storage device 172, such 
as a DVD drive, a floppy disk drive or a CD-ROM drive. 

[0085] In other words, the computer 1 6 (the watermarking unit 2, the server 1 2 or 

the detector 4) employs a general computer arrangement having a communication 
function. 

[0086] The server 12 supplies, as needed, image data Vin, watermark data W and 

key data Key, or an arbitrary combination of these data, to the watermarking unit 2 via 
the network 10 or via a storage medium 14, such as a DVD, a CD-ROM or a floppy disk. 

[0087] Fig. 3 is a diagram showing the arrangement of a watermarking program 

20, for carrying out the watermark data embedding process of this invention, that is 
executed by the watermarking unit 2. 

[0088] The watermarking unit 2 receives the watermarking program 20 from the 

server 12, via the network 10 (Fig. 1), or from the storage medium 14, via the storage 
device 172, and executes it. Thus, the watermarking unit 2 embeds watermark data W, 
which represents the copyright information, in the image data Vin that are supplied by the 
server 12, via the network 10, or by the storage medium 14, and obtains watermarked 
image data Vw (Fig. 3) in which the watermark data are embedded. 

[0089] The watermarking unit 2 transmits the obtained image data Vw to the 

detector 4 or to the server 12 via the network 10, or stores the image data Vw on the 
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storage medium 14 using the storage device 172 and then supplies it to the detector 4 or 
the server 12. 

[0090] As is shown in Fig. 3, the watermarking program 20 comprises: an image 

divider 200, orthogonal transformation units 202 and 24, a pattern embedding unit 204 5 
an output unit 206, a power element calculator 208, a basic pattern processor 22, a basic 
pattern adjustment unit 26, and a watermark pattern generator 28. 

[0091] The basic pattern adjustment unit 26 is constituted by an adjusted basic 

pattern calculator 262, an adjustment coefficient storage unit 260 and a power element 
" calculator 264. 

[0092] Figs. 4A and 4B are specific diagrams showing the image division process 

performed by the image divider 200 in Fig. 3. In Fig. 4 A, image blocks Iij are shown as 
the image division results, and in Fig. 4B, the structure of each image block (object image 
data) Iij is shown. In Fig. 4, to simplify and clarify the explanation, the number of pixels 
used to constitute an image is much smaller than the number that is actually required. 

[0093] Based on the structure (H, W) of image data Vin (one set of image data) 

and the structure (hb, wb) of the image blocks Iij, which are designated and input by a 
user via the input device 168 (Fig. 2), as is shown in Fig. 4A, the image divider 200 (Fig. 
3) divides the input image data Vin, consisting of H W pixels, into s x t image blocks II 1 
to Iij to 1st (s = H/hb, t = W/wb, 1 [[<]] i [[<]] s, 1 [[<]] j [[<]] t), each of which is 
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constituted by hb x wb pixels, as is shown in Fig. 4B. The obtained sxt image blocks 
are then output to the orthogonal transformation unit 202. 

[0094] The orthogonal transformation unit 202 performs the DFT transformation 

shown in equation 1 for the image blocks Iij that are received from the image divider 200, 
and outputs the obtained DFT coefficients Fij (also referred to, hereinafter, as image 
blocks Fij) to the pattern embedding unit 204 and the power element calculator 208. 



[0095] Equation 1 

I'ij-F(Iij) ....(1) 

[0096] wherein F denotes a Fourier transformation. 

[0097] The power element calculator 208 adds together the squared power values 



of the real element (Re) and the imaginary element (Im) of each of the image blocks Fij 
received from the orthogonal transformation unit 202. The power element calculator 208 
calculates the square root [(Re 2 + Im 2 ) 1/2 ] of the sum provided by the addition to obtain 
the power element T'ij of hb wb (also referred to, hereinafter, as an image block I"ij), and 
outputs the obtained power element to the watermark pattern generator 28. 

[0098] Fig. 5 is a diagram showing the structure of the basic pattern processor 22 

in Fig. 3. 



[0099] As is shown in Fig. 5, the basic pattern processor 22 comprises a grouping 

unit 220, a bit correspondence unit 222, a basic pattern storage unit 224, and a basic 
pattern selector 226. 
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[00100] Based on the above described image constituent data and the key data 
Key, which are set by a user at the input device 168 (Fig. 2) and are used to embed and 
detect watermark data, the basic pattern processor 22 employs these components to 
uniquely correspond the image blocks Iij with a bit wh (constituent data; 1 [[<]] h [[<]] g) 
extracted from the g (g is an arbitrary integer) bits wl to wg that constitute the watermark 
data w indicating additional information, such as copyright information for the image 
data Vin, and with the basic patterns Phaij and Phbij (e.g., the bit wh) that are included in 
the basic patterns PI to Pm (m [[>]] 2) and that correspond to the values 0 and 1 of the bit 
wh. 

[00101] As is shown in Fig. 4B, the basic patterns PI to Pm have the same hb wb 
structure as has the image block Iij. 

[00102] Actually, in accordance with the value of the bit wh that corresponds to 
each image block Iij, one of the basic patterns Phaij and Phbij is output by the basic 
pattern processor 22 as a selected basic pattern Phij that corresponds to the image block 
Iij. 

[00103] Fig. 6 is a diagram showing the image blocks Iij grouped using the 
grouping unit 220 in Fig. 5. In Fig. 6, the watermark data W are constituted by 10 bits, 
all the image blocks Iij are included in specific groups (not all the image blocks Iij need 
always be included in specific groups), and the structures in Figs. 4 A and 4B are 
employed for the image data Vin and the image block Iij. 
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[00104] As is shown in Fig. 6, based on the above image constituent data, the 
grouping unit 220 prepares, in the image data Vin, groups SI to Sg, each of which 
includes the same number of image blocks Iij that do not overlap each other and are 
distributed as evenly as possible. The grouping unit 220 then transmits the data 
representing the groups SI to Sg to the bit correspondence unit 222. 

[00105] The bit correspondence unit 222 (Fig. 5) corresponds each of the groups 
SI to Sg (Fig. 6) with one of the bits wl to wg of the watermark data w (for the easiest 
example, bit wh for Sh (1 [[<]] h [[<]] g)), and transmits the results to the basic pattern 
selector 226. 

[00106] The basic pattern storage unit 224 stores m types (m 2) of basic patterns 
PI to Pm, which are input by a user via the input device 168 or which are received from 
the storage device 172, and outputs them to the basic pattern selector 226. 

[00107] As is shown in Fig. 4B, the basic patterns PI to Pm are pixel patterns 
having different hb x wb pixel structures (Pb g [&]] Pq (p [[?]] q, 1 [[<]] p, q [[<]] m)). 
For example, the basic patterns PI to Pm are stored in advance in the basic pattern 
storage unit 224, or are generated from data held by the basic pattern storage unit 224, by 
using the key data Key and the watermark data W, or from data input to the basic pattern 
storage unit 224 by a user, via the input device 168 or the storage device 172. 

[00108] When the types m of the basic patterns are reduced in accordance with the 

bit count g of the watermark data, while taking into account the detectability and the 
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volume of the watermark data that is processed, the performance of the image processing 
system 1 is enhanced. 



[00109] The basic pattern selector 226 (Fig. 5) corresponds the bit wh of the 
watermark data w, which was corresponded with the group sh by the bit correspondence 
unit 222, with the two different basic patterns Pha and Phb (Pha g [[^]] Phb) of the basic 
patterns PI to Pm that are received from the basic pattern storage unit 224. 

[00110] Furthermore, for example, when the value of the bit wh is 0, the basic 
pattern selector 226 selects the basic pattern Phaij, or in another case selects the pattern 
Phbij, so that the same basic pattern Ph is corresponded with all the image blocks Iij that 
are included in the group Sh. 

[00111] As the selected basic pattern Phij that is corresponded with each of the 
image blocks Iij, the basic pattern selector 226 outputs correspondence results to the 
orthogonal transformation unit 24. 

[00112] The orthogonal transformation unit 24 (Fig. 3) performs the DFT process 
shown in equation 2 for the selected basic pattern Phij that is input by the basic pattern 
processor 22, and outputs the obtained DFT coefficient P'hij (referred to, hereinafter, 
simply as a selected basic pattern P'hij) to the basic pattern adjustment unit 26 and the 
watermark pattern generator 28. 

[00113] Equation 2 



P'hij = F(Phij) 
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[00114] The basic pattern adjustment unit 26 adjusts the value of the elements in 
the selected basic pattern P'hij of each image block Iij, an generates n types of adjusted 
basic patterns P'hij 1 to P'hijn and outputs them to the basic pattern adjustment unit 26. 

[00115] In the basic pattern adjustment unit 26, the adjustment coefficient storage 
unit 260 stores n types of adjustment coefficients CI to Cn, which are predetermined, 
which are input by a user via the input device 168 (Fig. 2), or which are generated by 
using the key data Key and the watermark data W. The adjustment coefficients CI to Cn 
are then output to the adjusted basic pattern calculator 262. 

[00116] The adjustment coefficients CI to Cn have the same hb wb structure as 
has the image block Iij in Fig. 4B, and each element has a value of 0 to 1 (it should be 
noted that, in accordance with the application, the value need not fall within this range). 

[00117] Fig. 7 is a diagram showing a method whereby the adjusted basic pattern 
calculator 262 in. 

[00118] Fig. 3 generates the adjusted basic patterns P'hij 1 to P'hijn. 

[00119] As is shown in equation 3, the adjusted basic pattern calculator 262 
multiplies the same elements in the adjustment coefficients CI to Cn and the selected 
basic pattern P'hij of the image block Iij that is received from the orthogonal 
transformation unit 24, and generates the n types of adjusted basic patterns P'hij 1 to P'hijn 
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(candidate data sets) of the hb wb structure. These n types of adjusted basic patterns 
P'hij 1 to P ? hijn are output to the watermark pattern generator 28. 



[00120] Equation 3 

[00121] For all the elements of Ck and P'hij, 
(P'hij[u])k = (C[u])kPWj[u] ....(3) 

where 1 [[<]] k [[<]] n and 1 [[<]] u [[<]] hb x wb; (P ! hij[u])k denotes the u-th element of 
k-th adjusted basic pattern; (C[u])k denotes the u-th element of the k-th adjustment 
coefficient; and P f hij[u] denotes the u-th element of the adjusted basic pattern. 

[00122] The power element calculator 264, as well as the power element calculator 
208 (Fig. 3), calculates the power elements P !, hijl to P"hijn (also referred to, hereinafter, 
as the adjusted basic patterns P"hijl to P"hijn) of the adjusted basic patterns P'hij 1 to 
P'hijn, and also calculates the power element P"hij (referred to, hereinafter, as the basic 
pattern P"hij) using the basic pattern P'hij that is received from the orthogonal 
transformation unit 24. The obtained power elements are then output to the watermark 
pattern generator 28. 

[00123] Fig. 8 is a diagram showing the arrangement of the watermark pattern 
generator 28 in Fig. 3. 

[00124] As is shown in Fig. 8, the watermark pattern generator 28 comprises 
entropy calculators 280 and 284, an adder 282, a comparator 286, a threshold value 
storage unit 288, selectors 290 and 296, a detectability calculator 292, and a supplemental 
pattern storage unit 294. 
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[00125] The watermark pattern generator 28 employs these components to obtain 
comparison result data Rl to Rn (variation indication data) that indicate the variations 
(the relative indiscernibility of the adjusted basic patterns) for the image block I"ij before 
and after the adjusted basic patterns P"hij 1 to P"hijn are added thereto. 

[00126] The watermark pattern generator 28 multiplies the elements in the adjusted 
basic patterns P m hijl to P nr hijln, which are determined to have less variation (are 
indiscernible after being embedded), and the same element of the basic pattern P'hij that 
is corresponded with the image block Iij. The watermark pattern generator 28 adds the 
multiplication values to obtain the inner products, and obtains the detectability data sets 
Dl to Din (data sets for indicating detectability) that represent the detectability of the 
adjusted basic patterns P m hijl to P m hijln. 

[00127] Furthermore, of the adjusted basic patterns P m hij 1 to P m hijln, the 
watermark pattern generator 28 selects, as the adjusted basic pattern that can be detected 
most easily, the adjusted basic pattern that provides the highest detectability data (that is 
the most easily detected). Then, the watermark pattern generator 28 outputs the pattern 
as the final watermark pattern P'hij to the pattern embedding unit 204 in Fig. 3. 

[00128] In the watermark pattern generator 28, the entropy calculator 280 
calculates the entropy e(I M ij), of the power element I"ij of the image block Iij, as shown in 
equation 4, and outputs it to the comparator 286. 

[00129] Equation 4 
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e(I M ij) = l/log(N) x S(-rulogru) .... (4-1) 
ru = wuTij[u]/Swmr4j[m] .... (4-2) 

where S denotes the accumulated addition value [[(X)]] within a range of u = 1 to N; u 
denotes the u-th element of Iij; N denotes the number of elements (32) included in I"ij; 
and wi denotes a predetermined constant. 

[00130] The adder 282 adds the image block I"ij and the adjusted basic patterns 
P"hijl to P"hijn, and outputs the obtained values (rij+P M hijl) to (I n ij+P"hijn) to the 
entropy calculator 284. 

[00131] The entropy calculator 284, as well as the entropy calculator 280, 
calculates the entropy for [e(I f, ij+P"hij 1) to e(I"ij+P"hijn)] for the addition values 
(rij+P"hijl) to (rij+P"hijn) that are received from the adder 282, and outputs the 
entropy to the comparator 286. 

[00132] The comparator 286 calculates absolute values |e(I"ij) - e(rij+P"hijl)| to 
|e(F'ij) - e(r'ij+P"hijn)| for the differences between the entropy (I"ij) for the image block 
I"ij and the entropy for [e(I"ij+P H hijl) to e(rij+P"hijn)] of the addition values 
(Fij+P"hijl) to (I"ij+P"hijn), which are received from the entropy calculator 280. The 
comparator 286 obtains the resultant comparison data sets Rl to Rn, which indicate the 
variations (the relative indiscernibility of the adjusted basic patterns) for an image before 
and after the watermark data are embedded therein by adding. The data sets Rl to Rn are 
then output to the selector 290. 

[00133] The threshold value storage unit 288 stores a threshold value e (0 [[<]] e) 

that is obtained, for example, through an experiment conducted in advance, is set by a 
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user via the input device 168 or the storage device 172 (Fig. 2), or is a threshold value e 
that is calculated by using the key data Key and the watermark data w. Thereafter, the 
threshold value e is output to the selector 290. 

[00134] The threshold value e is so set that when the value of the comparison data 
(1 [[<]] k [[<]] n) is smaller than the threshold value e, even when the adjusted basic 
pattern P'hijk that is corresponded with the data Rk is added to the image block I'ij and 
reverse transformation is performed for the resultant image block I'ij, a normal person 
can not or can only faintly perceive the displayed pattern, and so that when the value of 
the comparison data Rk (1 [[<]] k [[<]] n) is equal to or greater than that of the threshold 
value e, a normal person can perceive the adjusted pattern P f hijk that is added to the 
image block I'ij. 

[00135] Fig. 9 is a diagram showing a method whereby the selector 290 in Fig. 8 

selects one of the adjusted basic patterns P'"hijl to P fM hijl that has only slight variations, 

even when the adjusted basic patterns P'hij 1 to P ! hijn are added to the image block Iij. It 

should be noted that 0 [[<]] 1 [[<]] n, and that when 1 = 0 there are no variation indication 

data that are equal to or smaller than the threshold value e. 

[00136] The selector 290 compares the threshold value e received from the 

threshold value storage unit 288 with the absolute values |e(I ,? ij) - e(I"ij+P"hijl)| to |e(I"ij) 

- e(Fij+P M hijn)| of the differences that are received from the comparator 286. The 

selector 290 then selects the adjusted basic patterns P m hijl to P m hijl that provides the 

absolute value |e(I"ij) - e(rij+P"hijk)| (1 [[<]] k [[<]] n) less than or equal to the 

threshold value e, i.e., the absolute value |e(I"ij) - e(I M ij+P M hijk)| that satisfies equation 5. 
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The selected adjusted basic pattern is thereafter output to the detectability calculator 292 
and the selector 296. 



[00137] 



Equation 5 



|e(I M ij) - e(rij+P"hijk)| e 



....(5) 



[00138] 



When there are adjusted basic patterns P'"hijl to 



P'"hijl received from the 



selector 290, as is shown in equation 6, the detectability calculator 292 multiplies the 
same elements in the adjusted basic patterns P'"hijl to P'"hijl and in the basic pattern 
P'hij, and adds the results to obtain the product sum (inner product). 

[00139] Equation 6 

Dk = (P m hijk)(P'hij) 

= S(P m hijk)[u]P'hij[u] ....(6) 
where 1 [[<]] k [[<]] 1; S denotes the accumulated addition values [[(Z)]] within a range 
of 1 to N; (P m hijk)[u] denotes the u-th element of P'"hijk; and P f hij[u] denotes the u-th 
element of P'hij. 

[00140] Furthermore, the detectability calculator 292 outputs to the selector 296, as 
the detectability data sets Dl to Dl that represent the detectability of the adjusted basic 
patterns P m hijl to P nt hijl, the inner products obtained using equation 6. 

[00141] When the adjusted basic patterns P m hijl to P M, hijl input by the selector 
290 are not present, the detectability calculator 292 transmits a notification to that effect 
to the selector 296. 
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[00142] When the detectability calculator 292 notifies the selector 296 that the 
adjusted basic patterns P ,tr hijl to P"'hijln input by the selector 290 are not present, the 
supplemental pattern storage unit 294 receives a supplemental pattern Ps 5 which is output 
as the final watermark pattern P'ij, from the input device 168 or the storage device 172 as 
a result of an operation performed by a user, or generates and stores the supplemental 
pattern Ps using the key data Key and the watermark data W. The supplemental pattern 
Ps is then output to the selector 296. 

[00143] Upon the receipt of the detectability data Dl to Din from the detectability 
calculator 292, the selector 296 selects one of the adjusted basic patterns P m hij 1 to 
P m hijln that is received from the selector 290 and that corresponds to the detectability 
data for the maximum value, and outputs the selected basic pattern as a watermark 
pattern P'ij to the pattern embedding unit 204 (Fig. 3). In another case, the selector 296 
outputs the supplemental pattern Ps received from the supplemental pattern storage unit 
294 to the pattern embedding unit 204 (Fig. 3) as the watermark pattern P ! ij. 

[00144] The pattern embedding unit 204 (Fig. 3) embeds the watermark pattern by 
adding together the same elements in each image block I f ij, received from the orthogonal 
transformation unit 202, and in the watermark pattern P'ij, received from the watermark 
pattern generator 28. The obtained image block (Fij + P'ij) is thereafter output to the 
output unit 206. 

[00145] The output unit 206 performs run-length coding, for example, for the 
image block (Fij+P'ij) received from the pattern embedding unit 204, and forms the 

JP19990130US1 34 



obtained data into a predetermined transmission format for generating embedded picture 
data Vw. The output unit 206 then transmits the picture data Vw to the detector 4 via the 
network 10, or distributes the picture data Vw to the detector 4 via the storage device 172 
and the storage medium 14. 

[00146] Fig. 10 is a diagram showing the arrangement of a detection program 40, 
which is executed by the detector 4 in Fig. 1, for carrying out the watermark data 
detection process according to the present invention. 

[00147] The detector 4 executes the watermark data detection program 40 that is 
supplied to the storage device 172 via the storage medium 14, or that is supplied by the 
server 12 via the network 10, and implements the watermark data detection process in 
accordance with the present invention. 

[00148] As is shown in Fig. 10, the watermark data detection program 40 
comprises an image divider 42, a watermark pattern processor 44, an inverse orthogonal 
transformation unit 48, and a watermark data detector 46. 

[00149] The watermark data detection program 40 employs these components to 
detect the watermark data W held by the picture data Vw, in which the watermark data W 
are embedded by the watermarking unit 2 (the watermarking process program 20). 

[00150] The image divider 42 performs the inverted process for the output unit 206 

of the watermarking program 20 (Fig. 3), i.e., the run-length decoding and the division 

process in Fig. 4A, for the image data Vw that are supplied by the watermarking unit 2 
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(watermarking program 20 in Fig. 3) via the network 10 or the storage medium 14. Thus, 
the image divider 42 obtains the image block rij+P'ij in which the watermark pattern is 
embedded (referred to, hereinafter, simply as the image block Fij+P'ij) and outputs it to 
the inverse orthogonal transformation unit 48 and the watermark data detector 46. 

[00151] The inverse orthogonal transformation unit 48 performs the inverse DFT 
(IDFT) process for the image block rij+P'ij received from the image divider 42, and 
generates and stores image data Vin 1 in the storage device 172 (Fig. 2), or displays it on 
the display device 166. 

[00152] The watermark pattern processor 44 employs key data Key, which is set 
by the user using the input device 168 (Fig. 2), as well as the basic pattern processor 22 in 
the watermarking program 20 (Fig. 3), and corresponds the image block I'ij+P'ij with the 
basic patterns Phaij and Phbij. 

[00153] Based on the correlation of the image block I'ij+P'ij and the basic patterns 
Phaij and Phbij, the watermark data detector 46 detects the value of the bit wh of the 
watermark data W that is embedded in the image block rij+P'ij. 

[00154] Since the same bit is embedded in a plurality of image blocks of the 
watermarked image data Vw, the same bit wh is detected in a plurality of image blocks, 
and a bit value obtained by a decision of the majority is regarded as the final bit wh, so 
that the reliability of the detected image data W is increased. 
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[00155] The overall operation of the image processing system 1 (Fig. 1) will now 
be described. 

[00156] Fig. 1 1 is a flowchart showing, for example, the processing (S20) of the 
watermarking program 20 in Fig. 3. 

[00157] Fig. 12 is a flowchart showing the processing (S40) of the watermark data 
detection program 40 in Fig. 10. 

[00158] First, the operation of the watermarking unit 2 will be described while 
referring to Fig. 1 1 . 

[00159] When the watermarking unit 2 (Fig. 1) activates the watermarking 
program 20 (Fig. 3), at step 200 (S200) the image divider 200 divides the image data Vin 
as is shown in Figs. 4A and 4B, and generates the image blocks Iij. 

[00160] The basic pattern processor 22 corresponds each of the image blocks Iij 
with the selected basic pattern Phij, as was explained, for example, while referring to Fig. 
6. 

[00161] The basic pattern storage unit 224 of the basic pattern processor 22 
generates and stores, as needed, the basic patterns PI to Pm. 

[00162] The threshold value storage unit 288, of the watermark pattern generator 
28, generates and stores the threshold value e as needed. 
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[00163] The supplemental pattern storage unit 294 of the watermark pattern 
generator 28, a needed, generates and stores the supplemental pattern Ps. 



[00164] At step 202 (S202), the watermarking program 20 (output unit 206) 
determines whether the embedding process has been terminated for all the image blocks 
Iij. When the process has been terminated, the output unit 206 generates the 
watermarked image data Vw, from the image block rij+P'ij, in which the watermark 
pattern is embedded, and distributes the image data Vw to the network 10 and the server 
12. 

[00165] At step 204 (S204), the orthogonal transformation units 202 and 24 select 
the next image block Iij to be processed and the selected basic pattern Phij. 

[00166] At step 206 (S206), the orthogonal transformation units 202 and 24 
perform DFT transformation for the image block Iij and the selected basic pattern Phij 
that are selected at step 204, and generates the image block I'ij and the selected basic 
pattern P'ij and the power element calculators 208 and 264 generate their power elements 
I n ij. Then, the basic pattern adjustment unit 26 generates the adjusted basic patterns 
P'hijl to P'hijn, as was explained while referring to Fig. 7. 

[00167] At step 208 (S208), the entropy calculator 280, the adder 282, the entropy 

calculator 284 and the comparator 286 (Fig. 8), of the watermark pattern generator 28 

(Fig. 3), respectively calculate the absolute values (Rl to Rn) of the differences between 

the entropy values for the image block I"ij and the individual values obtained by adding 

the image block I"ij and the adjusted basic patterns P'hijl to P'hijn. 
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[00168] At step 210 (S210), the selector 290 (Fig. 8) extracts the difference 
(comparison data Rk) between the entropy of the image block Fij, to be compared next, 
and the entropy of the I n ij+P ,! hijk (1 [[<]] k [[<]] n), and determines whether the value of 
the data Rk is equal to or smaller than the threshold value e that is received from the 
threshold value storage unit 288. When the value of the data Rk is equal to or smaller 
than the threshold value e, program control advances to step 212, and when the value of 
the data Rk is greater than the threshold value e, program control is shifted to step 214. 

[00169] At step 212 (S212), the selector 290 adds the adjusted pattern P'hijk, 
which corresponds to the data Rk, to the process targets (candidate: adjusted basic 
patterns P nf hijl to P fM hijln) of the detectability calculator 292 and the selector 296. 

[00170] At step 214 (S214), the selector 290 determines whether all the n types of 
adjusted basic patterns P'hij 1 to P f hijn have been processed. When the process has been 
completed, program control advances to step 216. But when the process has not been 
completed, program control returns to step 210. 

[00171] At step 2 1 6 (S2 1 6), the detectability calculator 292 determines whether 
the adjusted basic patterns P"'hijl to P m hijln (candidates) to be processed have been 
received from the selector 290. When the adjusted basic patterns P'"hijl to P m hijln have 
been received, program control advances to step 220. But when these basic patterns have 
not yet been received, program control shifts to step 218. 
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[00172] At step 218 (S21 8), the detectability calculator 292 notifies the selector 
296 that there is no processing target, and upon the receipt of this notification, the 
selector 296 outputs, as the watermark pattern P'ij, the supplemental pattern Ps input by 
the supplemental pattern storage unit 294. 

[00173] At step 220 (S220), the detectability calculator 292 calculates, as 
detectability data sets Dl to Din, the inner products of the selected basic pattern P'hij and 
the individual adjusted basic patterns P m hijl to P m hijln that are received from the selector 
290. The selector 296 outputs, as the watermark pattern P ! ij, one of the adjusted basic 
patterns P fn hijl to P m hijln that of the detectability data Dl to Din provides the maximum 
value. 

[00174] At step 222 (S222), the pattern embedding unit 204 (Fig. 3) adds and 
embeds the watermark pattern P ? ij, which is received from the selector 296 (Fig. 8) of the 
watermark pattern generator 28, to the image block I'ij that is received from the 
orthogonal transformation unit 202. Program control then returns to step 202. 

[00175] As is shown in Fig. 12, when the detector 4 (Fig. 1) activates the 
watermark data detection program 40 (Fig. 10), at step 400 (S400) the image divider 42 
divides the watermarked image data Vw that is supplied by the watermarking unit 2, via 
the network 10 and the server 12 or via the storage medium 14, and obtains the image 
blocks Tij+P'ij. The watermark pattern processor 44 then employs the key data Key to 
correspond the image block I'ij+P'ij with the basic patterns Phaij and Phbij. 
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[00176] In accordance with a user's manipulation of the input device 1 68 (Fig. 2), 
the inverse orthogonal transformation unit 48 performs the IDFT transformation for the 
image block Fij+P'ij, and displays the result on the display device 166. 

[00177] At step 402 (S402) 5 the watermark data detector 46 selects the image 
block Fij+P'ij that is to be processed next. 

[00178] At step 404 (S404), the watermark data detector 46 obtains the correlation 
of the image block Fij+P'ij with the basic patterns Phaij and Phbij, and employs the 
correlation to determine the value of the bit wh of the watermark data W that is 
embedded in the basic patterns Phaij and Phbij. 

[00179] At step 406 (S406), the watermark data detector 46 determines whether 
the detection process has been completed for all the image blocks Fij+P'ij. When the 
detection process has been completed, program control advances to step S408. But when 
the detection process has not been uncompleted, program control returns to step 402. 
[00180] At step 408 (S408), the watermark data detector 46 examines the validity 
of the majority of the individual bits wh of the detected watermark data W. The 
watermark data detector 46 then combines the detected bits wh and outputs them as the 
watermark data W. 

[00181] As is described above, the data processing apparatus and the method 
therefor of the present invention are provided to resolve the conventional shortcomings. 
According to the invention, watermark data that indicate additional information, such as 
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copyright information, can be optimized in accordance with the contents of object data, 
and the optimized watermark data can be embedded. 

[00182] Furthermore, according to the image processing apparatus of the invention 
and the method therefor, watermark data can be optimized in accordance with the quality 
of image data, such as the property of an object, so that watermark data that is less 
noticeable and is easily detectable can always be employed to perform embedding. 

[00183] While the invention has been particularly shown and described with 
respect to preferred embodiments thereof, it will be understood by those skilled in the art 
that the foregoing and other changes in form and details may be made therein without 
departing form the spirit and scope of the invention. 
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DATA PROCESSING APPARATUS, IMAGE PROCESSING 
APPARATUS, AND METHOD THEREFOR 

ABSTRACT OF THE INVENTION 

[00184] It is on e obj e ct of th e inv e ntion to optimiz e w Watermark data that indicate 
additional information, such as copyright information, are optimized in accordance with 
the contents of object data, and , for e xample, to e mb e d the watermark data is embedded 
in image data. An image divider 200 divides image data Vi into image blocks Iij, and 
corresponds a selected basic pattern Phij with each of the image blocks Iij. Orthogonal 
transformation units 24 and 202 generate DFT coefficients Fij and P ! hij, and power 
element calculators 208 and 264 generate power elements I"ij and P"ij. A basic pattern 
adjustment unit 26 adjusts the coefficient of P'hij to generate P'hijl to PTiijn. A 
watermark pattern generator 28 calculates variations before and after P'hijl to PTiijn are 
embedded in I"ij, and selects, from P'hijl to PTiijn, P m hijl to P"Tiijl that provide 
variations equal to or smaller than the threshold value e. Then, the watermark pattern 
generator 28 selects, as a watermark pattern P f ij, the pattern that is most easily detectable, 
and for e mb e dding, a pattern embedding unit 204 that adds P'ij to I'ij. 
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